\section{Registry Service}
\label{sec:registryService}
\index{Service!Registry}

Package: \texttt{org.glite.rgma.server.services.registry}

The registry service has a registry instance for each VDB hosted
locally and a RemoteRegistry singleton to deal with forwarding
requests to remote registry services. Each instance is responsible for
dealing with its own replication and makes use of a RegistryDataBase
interface. An implementation of this interface is needed for each
supported RDBMS.



\subsection{Registry Instance}
\label{sec:registryInstance}

The principal components of the Registry Instance are shown in the picture at the
start of this chapter. When the instance is created, it creates a new instance of
a Registry Database and creates and starts a new Registry Cleanup and Replication
thread.  The instance has a status (\texttt{OFFLINE}/\texttt{ONLINE}) set to
\texttt{OFFLINE} when it is created. The registry instance is set to
\texttt{ONLINE} when the initialisation is complete.

All operations other than those associated with replication  have the same
signature as those in the Registry Service except that the vdbName is omitted.
The first part of all operations is to check the caller is authorized to call
this operation.

\subsubsection{Registry Replication}
\label{sec:registryReplication}
\label{sec:replicationIntroduction}
The Registry Instance is updated either by direct calls or upon
receipt of replication messages. A registry ``owns'' those records
that were last changed by direct calls and is responsible for pushing
updates of these records to other registries within the same
VDB. Records have an \texttt{isMaster} flag that \texttt{isMaster}
flag is set by any direct add request
(\texttt{getMatchingProducersForTables} or
\texttt{registerProducerTable}) and is cleared by a replication
message.

Records have a \texttt{lastContactTimeSec} which controls the purging
of old records however it is also used to detect a delayed replication
message. This is set by the registry receiving a direct call and is
subsequently replicated as part of the record.

Direct change requests are always respected, whereas replicated change
requests are ignored if they are out of date (i.e. before the
\texttt{lastContactTimeSec} associated with the record). 

If a registry should be unavailable then direct update requests will
get routed to a different registry instance and records in the new
registry will get the \texttt{isMaster flag} set. The original
registry will initially be unaware that another registry has
mastership of that record. When it receives replication messages from
the new master the \texttt{isMaster flag} will  be cleared.

The algorithm may result in more than one registry thinking that it is
master - however this is not a problem. It also possible to have no
master as the master may go down. However in all cases the system will
clean up when a registry assumes mastership for the record and
replicates its records.

If the clock on a registry is running too far behind time its
replication messages will be ignored if the receiving registry already
has an entry. If the clock is running fast then the replication
message will be received and the \texttt{lastContactTimeSec} will be
set in the future which will delay the purging of this record if it is
not explicitly deleted. This may result in non-existent resources
being returned by the registry.

Each \texttt{VDB} registry instance should log if the
\texttt{lastContactTimeSec} for a record is ahead of the current
time.

Note that where registry entries don't already exist, updates are
treated as inserts by the Registry Database and deletions of entries
that no longer exist are handled silently.

\paragraph{Full updates}

The algorithm described above assumes that all replication messages
are transmitted and processed correctly. Time stamps are associated
with the replication messages so that if a message has been lost the
recipient will know when the message is received. In this case a full
update is transmitted. This is generated by asking the Registry
Database for all records with the \texttt{isMaster} flag set.

\paragraph{The RegistryReplicationTable}

A hash table is used to hold the addRegistration and
deleteRegistration requests keyed on the primary key of the resource
entry. This means that if several addRegistration requests are
followed by a deleteRegistration request only the delete will be held.

When a request for a resource registration or deletion is received by
the Registry Instance, then in addition to passing the request to the
Registry Database it is inserted in the hash table.

When the Registry Instance replicate() operation is requested by the
Registry Cleanup and Replication a new hash table is created to take
new entries and the old one can be processed. This should be achieved
by a synchronisation lock between the call which adds to the hash
table and the code which switches the references.

A message is created from the hash table to be sent asynchronously to
all replicas using the \texttt{addReplica} call. If at the beginning
of the next cycle any messages have not been sent they are purged and
a full update will be sent instead to those replicas.

If the registry is restarted it will lose its hash table. When it
starts it must send a full update to all replicas. If the registry has
been down for some time other registries will have taken mastership
and most records will be ignored as they will be too old.

\paragraph{The addReplica message}

The addReplica message contains UTC timestamps of the last replication
cycle for this instance (\texttt{lastReplicationTime}) and the current
one (\texttt{currentReplicationTime}). This allows the recipient to
recognise if it has missed a message and to request a full update.

For a full update the \texttt{lastReplicationTime} is set to 0 so that the 
recipient will accept the message as the latest.

This is followed by an add section containing entries to add and a
delete section with keys to be deleted.

\subsubsection{Replication Operations}
This code is intended to create an \texttt{addReplicaMessage} or
\texttt{fullUpdateAddReplicaMessage} only when needed. If a remote
registry is down for a while repeated attempts will be made to send it
an \texttt{addReplicaMessage}. Once it comes back and receives a
message it will respond with \texttt{RGMAReplicaUpdateException} and
next time will get a \texttt{fullUpdateAddReplicaMessage}.

Note that the replicate call has a boolean flag \texttt{full} which is
set on the first call to request a full update.

Note also that when an \texttt{addReplica()} call is sent it is sent
asynchronously to another replica of this VDB. It does not use the
normal remote call mechanism via the \texttt{RemoteRegistry} object
but sends directly to the specified replica.



\subsection{Registry Database}

The RegistryDatabase is an interface to implementations for different
vendor specific databases with all of the SQL needed to read and
update it. The implementations uses vendor specific connection
wrappers in org.glite.rgma.server.services.registry. The database
table definitions and the methods in this component are described
here.


\subsection{RemoteRegistry}
\label{sec:RemoteRegistry}
\index{Class!RemoteRegistry}

This is the API used to forward calls to another Registry Service. It
has exactly the same interface as Registry Service but makes no use of
the authz context..  It uses the Remote Call Subsystem described in
section~\ref{sec:remoteCallSubsystem}. 

Each VDB which is not supported locally has a current registry service defined
to which calls will be forwarded. If the current registry is not defined all
services are pinged in parallel and the first to reply is selected as the
current registry. If it turns out that the current registry is set but not
reachable the current registry is cleared and a new one is sought.